Search for: All records

This work-in-progress research paper describes a study of different categorical data coding procedures for machine learning(ML) in engineering education. Often left out of methodology sections, preprocessing steps in data analysis can have important ramifications on project outcomes. In this study, we applied three different coding schemes (i.e., scalar conversion, one-hot encoding, and binary) for the categorical variable of Race across three different ML models (i.e., Neural Network, Random Forest, and Naive Bayes classifiers) looking at the four standard measures of ML classification models (i.e., accuracy, precision, recall, and F1-score). Results showed that, in general, the coding scheme did not affect predictive outcomes as much as ML model type did. However, one-hot encoding – the strategy of transforming a categorical variable with k possible values to k binary nodes, a common practice in educational research – does not work well with a Naive Bayes classifier model. Our results indicate that such sensitivity studies at the beginning of ML modeling projects are necessary. Future work includes performing a full range of sensitivity studies on our complete, grant-funded project dataset that has been collected, and publishing our findings. 

STEM undergraduate instructors teaching remote courses often use traditional lecture-based instruction, despite evidence that active learning methods improve student engagement and learning outcomes. One simple way to use active learning online is to incorporate exploratory learning. In exploratory learning, students explore a novel activity (e.g., problem solving) before a lecture on the underlying concepts and procedures. This method has been shown to improve learning outcomes during in-person courses, without requiring the entire course to be restructured. The current study examined whether the benefits of exploratory learning extend to a remote undergraduate physics lesson, taught synchronously online. Undergraduate physics students (N = 78) completed a physics problem-solving activity either before instruction (explore-first condition) or after (instruct-first condition). Students then completed a learning assessment of the problem-solving procedures and underlying concepts. Despite lower accuracy on the learning activity, students in the explore-first condition demonstrated better understanding on the assessment, compared to students in the instruct-first condition. This finding suggests that exploratory learning can serve as productive failure in online courses, challenging students but improving learning, compared to the more widely-used lecture-then-practice method. 

This WIP paper presents new research on exploratory learning, an educational technique that reverses the order of standard lecture-based instruction techniques. In exploratory learning, students are presented with a novel activity first, followed by instruction. Exploratory learning has been observed to benefit student learning in foundational math and science courses such as calculus, physics, and statistics; however, it has yet to be applied to engineering topics such as programming. In two studies, we tested the effectiveness of exploratory learning in the programming unit of a first-year undergraduate engineering course. We designed a new activity to help students learn about different python error types, ensuring that it would be suitable for exploration. Then we implemented two different orders (the traditional instruct-first versus exploratory learning’s explore-first) across the six sections of the course. In Study 1 (N=406), we did not detect a difference between the instruct-first and explore-first conditions. In Study 2 (N=411), we added more scaffolding to the activity. Students who received the traditional order of instruction followed by the activity scored significantly higher on the assessment. These findings contradict the exploratory learning benefits typically shown, shedding light on potential boundary conditions to this effect. 

This study examined the difficulty introduced by spaced retrieval practice in Calculus I for undergraduate engineering students. Spaced retrieval practice is an instructional technique in which students engage in multiple recall exercises on the same topic with intermittent temporal delays in between. Spacing out retrieval practice increases the difficulty of the exercises, reducing student performance on them. However, empirical research indicates that spaced retrieval practice is associated with improvements in students’ long-term memory for the retrieved information. The short-term costs and long-term benefits of spaced retrieval practice is an example of desirable difficulty, when more difficult exercises during the early stages of learning result in longer-lasting memory [1]. With support from the National Science Foundation (NSF), we sought to address: Does spacing decrease performance on retrieval practice exercises in an engineering mathematics course? Results showed that student performance was significantly lower for questions in the spaced condition than questions in the massed condition, indicating that we successfully increased the difficulty of the questions by spacing them out over time. Future work will assess final quiz performance to determine whether spacing improved long-term course performance, i.e., whether the difficulty imposed by spacing was desirable.